1. Field of the Invention
This invention relates generally to motion compensation in imaging systems. More particularly, it relates to improved techniques which electronically compensate for the relative motion of an image of a scene with respect to an electro-optical imaging array suitable for carriage by a reconnaissance vehicle.
2. Background Art
Most people who have attempted to photograph a rapidly moving object at close range with a simple box camera have found that the film image of the object is blurred or smeared due to the relative motion of the image over the film. The same effect is observed if the camera is moved during exposure of a stationary scene. If light conditions permit very short exposure times, the image essentially can be "stopped" by increasing shutter speed, and the smearing can be minimized.
Reconnaissance cameras frequently are required to record images in light conditions that prevent sufficiently short exposure times to eliminate image smearing by increases in shutter speed alone. Typically, such cameras are carried by aircraft for recording terrain scenes. Imaging systems used in such cameras comprise not only film, but also electro-optical devices, including charge-coupled devices. In any such camera in which relative motion exists between a scene to be imaged and the imaging system, the recorded image of the scene will be smeared unless some technique is used to compensate for the relative motion. Such techniques commonly are known as "forward motion compensation" or "image motion compensation." Left uncompensated, the smearing and image degradation resulting from the relative motion reduces the information content of the recorded image.
When a scene of interest is directly below an aircraft, the rates of motion of all points of the scene image in the field of view are nearly the same, and the motion can be compensated to avoid smear relatively easily. For example, if the imaging system is film, smear is avoided by moving the film emulsion at the same rate and in the same direction as the motion of the scene image in the focal plane of the camera.
When the scene of interest is not directly below the aircraft, but is perpendicular to the direction of flight and at an oblique angle, the solution to the problem of image motion compensation becomes difficult, principally because objects at a closer distance to the aircraft appear to be moving faster relative to the aircraft than objects farther away. Similarly, when the scene of interest is forward of the aircraft, the solution to the problem of image motion compensation becomes more difficult because terrain farther ahead appears to be moving slower than terrain closer to the aircraft.
The specifics of the problem are modified when changes in the aircraft velocity, height above ground, or camera depression angle occur. These changes affect the rate of image motion in the focal plane of the camera, and they must be taken into account by a forward motion compensation system. The extent of image smear is most directly a function of the aircraft velocity relative to the ground (V), the height above ground (H), and the time period of exposure. The extent of the image smear is also a function of the magnitude of the field of view and the angle of depression below the horizontal where the field of view is located.
Mechanically-based forward motion compensation schemes have been devised and implemented in an attempt to eliminate image smear due to forward motion, or to reduce such smear to acceptable levels. Such schemes have been implemented by use of a translating film, a translating lens, or a rotating mirror.
In the translating film technique, the film is moved in the same direction and velocity as a portion of an image. The image motion velocity (V.sub.i) and the film velocity (V.sub.f) are made essentially synchronous and relative motion between them during the exposure time period essentially is eliminated. The net result is that the image portion is essentially stationary with respect to the film during the exposure time period. The translating film technique is frequently used on short and medium focal length framing type cameras.
In the translating lens technique, if a lens is translated in space, the image of distant objects will translate with the lens in a one-to-one relationship in the same direction. Therefore, if the lens in an aerial camera is translated at the proper velocity opposite to the direction of flight, the image velocity caused by the aircraft forward motion is cancelled by the image velocity due to the moving lens. The net result is that the image is essentially stationary relative to the film, and therefore no essential motion smearing is occurring during the exposure. This type of forward motion compensation is frequently used on short and medium focal length panoramic type scanning cameras. See, e.g., Ruck, Design Versatility of the Prism Panoramic Camera: The KS-116 and KA-95Cameras, SPIE Proceedings, Vol. 309, paper 309-10, (Aug. 27-28, 1981).
In the rotating mirror technique, as the aircraft is flying in a given flight path, the objects in the scene have an apparent angular velocity relative to the camera. The apparent angular velocity is related to the aircraft velocity and the range to the target. If a camera is looking into a mirror at a nominal angle of 45.degree., the camera line of sight is deviated by a nominal 90.degree. angle. If the mirror is rotated in the proper direction and at the proper rate during the exposure, the scene appears to have no motion relative to the camera. Therefore, at the film plane, the image is essentially stationary and forward motion image smear is substantially negated. The rotating mirror forward motion compensation concept is often used on long focal length frame and line scanning type cameras.
All three of the foregoing mechanical forward motion compensation schemes are employed in various aerial reconnaissance cameras, including film cameras and electro-optical line scanning cameras. A principal disadvantage of these forward motion compensation schemes is that they all involve mechanical devices and consequently add complexity, weight, and expense to the imaging system.
Other methods and techniques of forward motion compensation also have been developed. In the Prinz patent, U.S. Pat. No. 4,505,559, a mechanical image motion compensation technique is disclosed wherein a slot on a focal plane shutter moves in a direction transverse to the direction of film transport, while the direction of film transport is made parallel to the direction of flight. In order to compensate for a component of image motion, the slotted shutter is coupled to an encoder which, in the course of making an exposure, reports to a computer for instantaneous field position of the shutter slot. The computer determines the speed profile in the interval between successive exposures and issues a control signal to regulate the film drive.
The Gordon et al. patent, U.S. Pat. No. 4,157,218, discloses a wide angle scanning reconnaissance camera which uses a wide angle lens and a curved exposure slit disposed adjacent an image receiving surface, typically film. The exposure slit is crescent shaped to correspond to equal points of equal range so that there is negligible differential image motion within the slit, and the slit is of constant width in the direction of vehicle motion. The image receiving surface (e.g., film) is driven in the direction of vehicle movement at a speed corresponding to the altitude-velocity ratio of the vehicle with respect to the object field.
The Wight patent, U.S. Pat. No. 4,908,705, discloses an electro-optical aerial reconnaissance system wherein a linear charge-coupled device imager is movably positioned in the focal plane of a fixedly mounted wide angle lens system. The linear imager is moved in the fore and aft direction in the same direction as the apparent motion of the image to reduce the apparent image motion and consequent smear. This system, however, only is designed for imaging terrain at nadir, aft of nadir or ahead of nadir, and does not provide forward motion compensation in a side-oblique reconnaissance application.
For aerial reconnaissance, electro-optical cameras, particularly those of the charge-coupled device variety, are perceived as superior to film cameras to an increasing extent. In an electro-optical camera, radiation from an image of interest impinges on a solid state device typically having several thousand (at least) picture elements or pixels. The incident radiation is converted into charge packets (pixel information) at the photosites (pixels) and is collected in potential wells. The charge packets contain scene information, and upon being transferred out of the device, are converted into electrical signals. The primary advantage of an electro-optical imaging camera is that the scene information can be almost instantaneously "downloaded" from a reconnaissance aircraft to an earth-based station, or can be converted to a video image. Since charge-coupled device imaging cameras have very small pixels closely spaced together, the resolution of a resulting image tends to be very high. Electro-optical imaging cameras can be made sensitive to particular frequencies of incident radiation. Background information on charge-coupled devices can be found in standard texts such as D. Schroder, Modular Series On Solid State Devices, Ch. 3, 4, Addison-Wesley (1987), and in C. Sequin and M. Tompsett, Charge Transfer Devices, Bell Telephone Laboratories, Academic Press (1975), and in S. M. Sze, Physics of Semiconductor Devices, Ch. 7, John Wiley & Sons, Inc. (1981).
In a linear electro-optical focal plane reconnaissance detector, such as the linear detector of the Wight patent, a scene of interest is scanned a line at a time across an array in a direction perpendicular to the array length. Because the means of scanning is provided by the aircraft forward motion, the aircraft must maintain a steady, well defined flight path while the scene is being recorded. Depending on the size of the scene, the recording time for any one target may range between 10 and 20 seconds, or even longer. In a military situation in which the reconnaissance aircraft may be subject to enemy threats, the vulnerability during the recording time may be excessive.
To reduce the time needed to image a scene of interest and thus reduce the time of exposure to hostile threats, a preferred mode of the present invention uses a two-dimensional electro-optical imaging area array, rather than a linear (one-dimensional) array. An area array can image an entire scene instantaneously, rather than a line at a time. Until recently, only relatively small electro-optical imaging arrays have been commercially available, and are typically used in television cameras. But large, high pixel count area arrays suitable for aerial reconnaissance sensors are now entering the realm of feasibility. Two scientific imagers used in astronomical applications, the Tektronix TK 1024 CCD and the Ford Aerospace 4,096.times.4,096 pixel element array, can be adapted to the present invention by subdividing the arrays in column groups and providing the circuitry for faster frame rates. Information useful for designing high pixel count area arrays also is found in J. Janesick, Multi-Pinned-Phase Charge-Coupled Device, NASA Tech. Brief Vol. 14, No. 8, Item No. 115, p. 22, Jet Propulsion Laboratory, August, 1990.
An area type detector array can convert an entire image of a scene into a complete frame of pixel information during a short exposure period, typically on the order of a hundredth of a second. After the exposure period, a shutter can be used to prevent continued exposure while the pixel information in the array is read-out to a signal processing unit. After the read-out is completed, the array is ready for the next frame exposure. If the frame read-out time is short (much less than a second), then consecutive frames can be taken in sub-second intervals in order to obtain large scene coverage in short periods of time. By providing motion compensation in an area detector having exposure time controlled by a shutter, the present invention substantially reduces exposure of an aircraft, a pilot and a detector array to enemy countermeasures.
The motion compensation techniques of the present invention also enable effective use of a detector array having a large number of photosites or pixels (e.g., four to six thousand or more in both the columns and rows of the array) that will image a large area of terrain in every frame. The present invention makes such arrays practical by preserving image resolution (i.e., scene detail information) in every frame of imagery.
The present invention also enables high array exposure sensitivity. That is, motion compensation is accomplished in a way that promotes long exposure time without blurring the image. In a pushbroom system, exposure time is limited by the line rate which is dictated by the aircraft velocity to height ratio (V/H). For the present invention the exposure time is not limited by the aircraft V/H. This permits operation at lower scene illumination levels and extends the available time of day for light-sensitive sensor operation.
Additionally, the present invention provides for rapid read-out of collected scene information. A rapid read-out of the electrical signals of an array is necessary in order to achieve high frame rates. High frame rates are desirable to allow multiple images to be recorded in a short time such as required for stereo imagery.
The present invention offers additional advantages over linear electro-optical sensors. For example, motion compensation is provided irrespective of whether an electro-optical imaging array is deployed in a side oblique, forward oblique, or down-looking application. The present invention also provides lower scene distortion and true stereo imagery capability. By using the present invention to replace a film camera mounted in an aircraft, motion compensation can be accomplished while retaining similar exposure times and aircraft maneuverability characteristics.
The present invention is reliable and robust because it requires no mechanical scanning mechanism, no rotating mirrors and no translating lenses in order to achieve forward motion compensation.
Another feature of the invention is that it is suitable for use in a wide variety of applications, such as tactical reconnaissance, drug interdiction, low intensity conflict, low and medium altitude missions, and reconnaissance at low light levels.